Mendeley readership as a filtering tool to identify highly cited publications

This study presents a large scale analysis of the distribution and presence of Mendeley readership scores over time and across disciplines. We study whether Mendeley readership scores (RS) can identify highly cited publications more effectively than journal citation scores (JCS). Web of Science (WoS) publications with DOIs published during the period 2004‐2013 and across 5 major scientific fields have been analyzed. The main result of this study shows that readership scores are more effective (in terms of precision/recall values) than journal citation scores to identify highly cited publications across all fields of science and publication years. The findings also show that 86.5% of all the publications are covered by Mendeley and have at least one reader. Also the share of publications with Mendeley readership scores is increasing from 84% in 2004 to 89% in 2009, and decreasing from 88% in 2010 to 82% in 2013. However, it is noted that publications from 2010 onwards exhibit on average a higher density of readership vs. citation scores. This indicates that compared to citation scores, readership scores are more prevalent for recent publications and hence they could work as an early indicator of research impact. These findings highlight the potential and value of Mendeley as a tool for scientometric purposes and particularly as a relevant tool to identify highly cited publications. Introduction and background Scholars use social media tools for different purposes, for example to collaboratively distribute scientific information, share knowledge and ideas, and communicate with their peers (Gruzd, Staves, & Wilk, 2012). Among the different altmetric sources, Mendeley is one of the most important online reference managers with more than 4 million users worldwide 1 , and is especially popular among students and postdocs (Zahedi, Costas, & Wouters, 2014b; Haustein & Larivière, 2014). Mendeley exhibits a high coverage of scientific publications, with coverage values higher than 60% or even 80% for WoS publications depending on the field (Costas, Zahedi, & Wouters, 2015b). Meaning of Mendeley readership Mendeley collects usage statistics per document as they are added by the different users to their private libraries. These statistics are commonly known as " readership statistics " , although in reality the metrics don't necessarily reflect the actual 'reading activity' by Mendeley users. For example, scholars do not necessarily always 'read' the scholarly outputs